1. Field of the Invention
The present invention relates to a system for transmitting and receiving data streams for forming 3-dimensional virtual space displayed by 3-dimensional computer graphics (referred to as xe2x80x9c3-DCGxe2x80x9d, hereinafter) based on a network such as the Internet and more particularly, to an apparatus for and a method of transmitting and receiving data streams representing a 3-dimensional skeletal structure so as to efficiently transmit and receive natural motions of a character having a complicated skeletal structure, e.g., a human being.
2. Description of the Prior Art
In recent years, virtual malls, electronic commerce and various relevant home pages on the Internet such as World-Wide Web (WWW) attract considerable attention as an applicable field of 3-DCG. Especially, thanks to rapid development of the Internet, the trend towards easy handling of relatively high-quality 3-DCG such as games and movies at home is gaining momentum. As shown in FIG. 8, a plurality of machines called xe2x80x9cclientsxe2x80x9d PCC1-PCCn, e.g., personal computers are connected to a machine called xe2x80x9cServerxe2x80x9d PCS, e.g., a personal computer or a workstation through the Internet in WWW or the like. By restructuring information such as sounds, texts and layout information downloaded from the server PCS as necessary, the information transmitted from the server PCS can be obtained by the clients PCC1-PCCn. Usually, a communication method based on Transmission Control Protocol/Internet Protocol (TCP/IP) is employed for communication between the server PCS and the clients PCC1-PCCn.
Conventionally, data supplied from the server PCS has been mainly restricted to text data and image data. However, recent progress of standardization of the Virtual Reality Modeling Language (VRML) and browser of the VRML has given an impetus to transfer of 3-DCG itself such as shapes and scenes. The VRML referred to above is described below-briefly. In a known data layout consisting mainly of images and texts as in the Hypertext Markup Language (HTML), tremendous time and cost are necessary for transferring image data, especially, data on moving picture. Therefore, in existing apparatuses, there is a limit to network traffic. On the other hand, in 3-DCG, all information including shapes, visual point information and light source information is processed in 3-dimensional data. Recent progress of computer graphics (referred to as xe2x80x9cCGxe2x80x9d, hereinafter) technology has brought about rapid improvement of quality of images produced by CG. Therefore, also from a standpoint of data quantity, it is quite efficient to transfer CG data itself.
Usually, even if a quantity of CG data is not more than {fraction (1/100)} of that of conventional image data, the CG data is capable of displaying images having a quality equivalent to that of the conventional image data. In other words, the CG data has a compression ratio which is not less than 100 times that of the conventional image data. Hence, in transmission of image data through a network, an attempt is being made to use CG data, especially, 3-DCG data as standard. As one example of this attempt, the above mentioned VRML is proposed (VRML Ver2.0). The VRML Ver2.0 stipulates data format of shape data called xe2x80x9cprimitivexe2x80x9d, light source data, visual point data, texture data, etc. and how to specify displacement of a rigid body. Such a data format is referred to as a xe2x80x9cVR formatxe2x80x9d (VRF) for the sake of convenience.
On the other hand, in conventional CG, attention is recently given to animation technology for forming images on real-time basis. By using this real-time animation technology, real motions of a CG character are reproduced mainly in commercial messages and movies. As an example, a complicated shape such as a human being is expressed by a skeletal structure and sophisticated motions of the skeletal structure can be reproduced naturally by defining motion amounts of joints of the skeleton, which change momentarily.
In conventional 3-DCG modeling languages on the Internet, mainly, the VRML, it has been impossible to impart motions on real-time basis to the shape having the complicated structure such as the human being. Meanwhile, it has also been impossible to formulate a system in which a plurality of users have virtual 3-dimensional space in common in virtual space involving a network such that their avatars are operated interactively in the common virtual 3-dimensional space. Furthermore, it has been impossible to transmit and receive voices or music on real-time basis synchronously with motions of the avatars in the common virtual 3-dimensional space.
Accordingly, an essential object of the present invention is to provide an apparatus for and a method of transmitting and receiving data streams representing 3-dimensional virtual space, in which motions can be imparted on real-time basis to a shape having a complicated structure such as a human being, a plurality of users have virtual 3-dimensional space in common in virtual space involving a network such that their avatars can be operated interactively in the common virtual 3-dimensional space and voices or music can be transmitted and received on real-time basis synchronously with motions of the avatars in the common virtual 3-dimensional space.
In order to accomplish this object of the present invention, an apparatus for transmitting and receiving data streams representing 3-dimensional virtual space, according to the present invention comprises: a transmitting member; a receiving member; and a transfer means for connecting the transmitting member and the receiving member in bidirectional manner; wherein a shape data stream, a motion data stream and an audio data stream representing a shape and a motion of a skeletal structure in 3-DCG and a sound synchronous with the motion, respectively are transmitted to the receiving member from the transmitting member such that the receiving member generates the 3-DCG through synchronization of the motion data stream and the audio data stream; the transmitting member comprising: a transmitting means for simultaneously transmitting, on real-time basis in response to a request from the receiving member, the shape data stream, the motion data stream and the audio data stream synchronous with the motion data stream as a plurality of streams in accordance with a format of a predetermined data stream such that the transfer means interactively transfers the streams from the transmitting member to the receiving member; the receiving member comprising: a data stream receiving means which receives the streams transmitted from the transmitting member via the transfer means so as to discriminate kinds and formats of the received streams and receives the streams as necessary on real-time basis so as to subject the discriminated streams to restoration processings required for the streams, respectively; a 3-dimensional virtual space generating means which generates the shape of the skeletal structure in the 3-dimensional virtual space or the 3-dimensional virtual space by using a shape stream of the streams received by the receiving member or 3-dimensional shape data read preliminarily and moves the shape of the skeletal structure by using a motion stream of the streams; and a sound generating means which reproduces, in case the streams contain an audio stream, the audio stream synchronously with the motion stream.
Meanwhile, a method of transmitting and receiving data streams representing 3-dimensional virtual space, in which a shape data stream, a motion data stream and an audio data stream representing a shape and a motion of a skeletal structure in 3-DCG and a sound synchronous with the motion, respectively are transmitted from a transmitting member to a receiving member such that the receiving member generates the 3-DCG through synchronization of the motion data stream and the audio data stream, according to the present invention comprises: a first step of simultaneously transmitting, on real-time basis in response to a request from the receiving member, the shape data stream, the motion data stream and the audio data stream synchronous with the motion data stream as a plurality of streams in accordance with a format of a predetermined data stream; a second step of connecting the transmitting member and the receiving member in bidirectional manner so as to transfer the streams from the transmitting member to the receiving member interactively; a third step of receiving the transferred streams so as to discriminate kinds and formats of the received streams and receiving the streams on real-time basis so as to subject the discriminated streams to restoration processings for the streams, respectively; a fourth step of generating the shape of the skeletal structure in the 3-dimensional virtual space or the 3-dimensional virtual space by using a shape stream of the streams received by the receiving member or 3-dimensional shape data read preliminarily and moving the shape of the skeletal structure by using a motion stream of the streams; and a fifth step of reproducing, in case the streams contain an audio stream, the audio stream synchronously with the motion stream.
By the above mentioned arrangement of the apparatus and the method of the present invention, the shape data of the skeletal structure such as a human being is transferred as the shape data stream, the fundamental motion of the skeleton is delivered as the motion data stream and the audio data accompanying the motion of the skeletal structure is transferred as the audio data stream together with information on synchronization and compression in each of the data streams and correspondence among the data streams.
Therefore, in accordance with the present invention, smooth motions of a character in a transmission and reception system based on a network and audio information synchronous with the motions can be transmitted and received interactively in response to a request of a user and quantity of the data to be transferred can be reduced greatly.